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Abstract: 

It is previously shown that proper random linear samples 
of a finite discrete signal (vector) which has a sparse rep- 
resentation in an orthonormal basis make it possible (with 
probability 1) to recover the original signal. Moreover, 
the choice of the linear samples does not depend on the 
sparsity domain. In this paper, we will show that the re- 
placement of random linear samples with deterministic 
functions of the signal (not necessarily linear) will not re- 
sult in unique reconstruction of fc-sparse signals except for 
k = 1. We will show that there exist deterministic non- 
linear sampling functions for unique reconstruction of 1- 
sparse signals while deterministic linear samples fail to do 
so. 

1. Introduction 

Recent results in compressed sensing [T| [3] |4j [7] show 
that for a class of finite discrete signals named sparse, 
it is possible to reconstruct the signal from significantly 
lower number of samples than predicted by Nyquist rate 
in uniform sampling theory. More precisely, if x„ x i 
is a vector whose representation in the orthonormal ba- 
sis \& = [ipij] n xn has at most k nonzero elements , 
it is called fc-sparse. Also s = ■ x is the transla- 
tion of x in the sparsity domain (since '3/ is orthonormal 
\& ,_1 = \I> ); i.e., s has at most k nonzero elements. 
It is proved that if m is of order not less than O (k ■ log ( f - ) ) , 
m random linear combination of the samples (multipli- 
cation by a random matrix <t>) of a /c-sparse signal with 
length n (k <C n) where the coefficients of the linear 
samples have normal i.i.d. distribution, suffice (with over- 
whelming probability) to reconstruct the signal |7|. Other 
distributions have also been proposed which yield the 
same result J5). One of the features of this type of ran- 
dom sampling which was originally of main concern (but 
no longer as important as it was) is that at the time of sam- 
pling it is unimportant in which basis the signal is sparse. 
The domain of sparsity comes into the picture only at the 
time of the reconstruction. Therefore, this type of sam- 
pling is not matched to a specific sparsity domain. The 
other advantage of the method lies in its reconstruction 
method. The mentioned order of m is the same as the 
order for which the minimization using £q norm for find- 
ing the sparse solution could be replaced with l\ without 
considerable change in the solution (solving a polynomial 



time problem instead of an NP-complete problem) Bl flOl . 
Since in most of the applications the input signal has a 
priory known characteristics, the sparsity domain is also 
known. Thus, it is logical to design a deterministic sam- 
pling matrix (<&); in this way, we eliminate the need to 
save the sampling matrix. Some possible matrices are in- 
troduced in ]6] and ifTTI . 

Although the analog branch of the compressed sensing is 
being developed parallel to the improvements in the dis- 
crete domain [8, 9], we will only consider the discrete 
case in this paper. We will investigate the consequences 
of deterministic sampling when the sparsity domain is not 
known at the time of sampling; however, we do not restrict 
the sampling functions to be linear. In fact we are looking 
for deterministic functions of x which uniquely represent 
all /c-sparse signals irrespective of their sparsity domain 
(for reconstruction the domain should be known). 

2. Preliminaries 

Before we start the main argument, it is necessary to point 
out a key issue: since the cardinality of C (or R) is the 
same as C n (or R"), there exists an injection g : C n — > C 
(or g : R™ — > R) which as a sampling function, com- 
presses the whole information of a vector into a single 
sample, even without the assumption of sparsity. Although 
the compression is amazing, the output sample can not 
be easily quantized; i.e., the theoretical solution is im- 
practical. For this reason, we are looking for sampling 
functions which employ the sparsity constraint in order 
to reconstruct the original signal and then we will dis- 
cuss how they can be quantized. To formulate the men- 
tioned constraint: we are looking for sampling functions 
fi : C n — ► C, 1 < i < m that result in equivalent sam- 
pling classes defined by: 



{xeC" | VKKm: /<(x) = Si } (1) 



such that in each arbitrary orthonormal basis, at most one 
of the elements of S has fc-sparse representation. More- 
over, the sparsity constraint should play a key role; i.e., 
for many cases we should have |«S| > 1. 
First we show why deterministic linear sampling is not a 
proper choice. Assume the samples are produced as: 
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If m > n, the number of samples exceeds the num- 
ber of original elements in x, which defeats the under- 
lying purpose of compression. For the case of m < n, 
rank(<l>) < m < n and therefore, the null space of <1> will 
have non-zero dimension: 

3veC", ||v|| £2 = 1 : * • v = (3) 

Let V 1 - be the subspace of C™ formed by the vectors per- 
pendicular to v: 

V^jxeC" | v H -x = 0}, (4) 

and let {m, . . . , u„_i} be an orthonormal basis for this 
subspace. Hence, {v, m, . . . , u„_i} forms an orthonor- 
mal basis for C n , or equivalently, 

* = [v, ui, . . . , u„_i] (5) 

is a unitary matrix. Obviously, the input vector v is 1- 
sparse with respect to the representation in , 3>; however, v 
lies in the null space of <I> and can not be recovered using 
its samples (all zero) produced by <&. 
Thus, for any fixed linear sampling set with less number 
of samples than the original signal's length, there exists 
an orthonormal basis and a 1 -sparse signal (in this basis) 
which can not be uniquely recovered from its samples. 

3. General Sampling for fc-sparse Signals 

Although deterministic linear sampling fails either in com- 
pression or reconstruction, there exists the possibility to 
introduce general nonlinear sampling functions which do 
not have these drawbacks. In this section we will show 
that for fc-sparse signals with k > 2 no such sampling 
functions exist. 

Let us assume that {/i(x)}™ 1 are the sampling functions 
and let {si}™ 1 be the respective samples obtained from 
a given fc-sparse vector x (in an orthonormal basis). Re- 
versibility of the sampling process is equivalent to the fact 
that xo is the only fc-sparse vector which results in samples 
{ s i}iLi ■ As previously defined, we represent the sampling 
equivalent class of the obtained samples {sj}™ 1 by S: 

S = {x G C™ | VI < i < m , fi(x) = Si } (6) 

Since x G S we know 5^0. Since the case m > 
n contradicts the compression concept, we are assuming 
m < n. Moreover, if for all input vectors xq G C™ we 
have |<S| = 1, sparsity constraint is of no use; i.e., a 1-1 
mapping from C™ to C m is formed which is undesired due 
to its quantization problem. Therefore, there exist input 
vectors for which the equivalent class S has at least two 
elements. We choose x among these input vectors; i.e., 
there exists xi ^ xo such that the samples obtained from 
both of x and xi are the same (x , xi G S). Let v be the 
normal vector in the direction of x and let vi be defined 
as follows: 

vi = xi - (xi, v )vo (7) 

where (a, b) stands for the inner product of the vectors a 
and b, ((a, b) = a H ■ b). We have two cases: 



1 . If Vi = 0, both xo and xi lie in the direction iden- 
tified by the normal vector vo. Similar to the ar- 
gument in the previous section, there exist normal 
vectors m, . . . , u n _i such that {v , Ui, . . . , u„_i} 
forms an orthonormal basis for C™. Obviously both 
xo and Xi are 1 -sparse in this basis and moreover, 
they produce the same set of samples using the as- 
sumed sampling functions. Hence, using the men- 
tioned samples, these two vectors can not be distin- 
guished even with the sparsity constraint. 

2. If vi ^ 0, we define vi to be the normal vector in 
the direction of vi . Since vo and vi are orthogonal, 
vo and vi are also orthogonal: 

(vi,v ) = (xi,v > - (xi,v ) • (v ,v ) =0 

= l|vo||? 2 =l 

=> vi _L v => vi -L v (8) 
Moreover from the definition of vi we have: 

xi = Vi + (xi,v )vo 

= (xi, vi)vi + (xi,v )v (9) 

Therefore, x and Xi are respectively 1 -sparse and 
2-sparse in any orthonormal basis which contains 
v and Vi (existence of such a basis is trivial 
by considering the subspace of C™ orthogonal to 
spanjvo) Vi}). Thus, if fc > 2, both xo and xi are 
valid solutions to the sparsity constraint. Thus, again 
we can not uniquely reconstruct the original signal 
from its samples. 

4. Sampling for l-sparse Signals 

The only case that we are left is the general (nonlinear) 
sampling of l-sparse signals. In fact we will show the 
existence of such sampling functions for this case, but first 
lets focus on the conditions that the sampling functions 
should satisfy: 

Lemma 1 The set S C C" is uniquely l-sparse decod- 
able iff: 

Va^bGS: 0<|<a,b>|<||a||* 2 -||b||/ a (10) 

where (a, b) is the inner product of the two vectors (a H ■ 
h), \\-\\e 2 is the ii norm of the vector {square root of the 
inner product of the vector by itself) and \ . \ refers to the 
absolute value of a complex number. 

Using the Cauchy inequality and the non-negative prop- 
erty of the absolute-value operator, it is straightforward 
that: 

Va,bGC": 0< |(a,b}| < ||a||, 2 • ||b||, 2 (11) 

The only characteristic which distinguishes an arbitrary 
set from a uniquely l-sparse decodable set is that the 
equalities do not occur for the latter. Now we get back 
to the proof of the lemma. According to the Cauchy theo- 
rem, for the right inequality we have: 

|(a,b)| = ||a||/ a • ||b|| /a 3cGC:a = cb (12) 



If S contains two unequal vectors a, b for which the above 
equality holds, in the orthonormal basis containing the 
normal vector in the direction of a (or b) both of the vec- 
tors are 1 -sparse (since they are in the same direction) 
which means that S is not uniquely 1 -sparse decodable. 
On the other hand, if for all unequal vectors of S strict 
inequality holds, each line in C" passing through the ori- 
gin can not intersect S at more than one point. This means 
that for each vector of an orthonormal basis, there exists at 
most one element of S which is 1 -sparse in this direction. 
The left inequality considers the orthogonality of the vec- 
tors in S: 

| (a, b) | =0 alb (13) 

Assume S contains two nonzero perpendicular vectors 
a, b and let a, b be the normal vectors in their directions, 
respectively. Since a and b are orthogonal, there exists an 
orthonormal basis for C™ which contains both a, b; obvi- 
ously a and b are 1 -sparse in this basis which implies that 
S is not uniquely 1 -sparse decodable. 
For the sufficiency of the condition, if no two vectors of S 
are perpendicular, among the vectors of each orthonormal 
basis of C™ there exists at most one which intersects S 
(two intersections reveal a perpendicular pair in S). Also 
the result of the right-side inequality restricts the number 
of intersections for each direction to one; so, at most one 
of the vectors in S can be 1 -sparse in this basis (1 -sparse 
decodablity of S) and the proof of the lemma is complete. 
Our next step is to introduce sampling functions whose 
equivalent sampling classes fulfill the condition of the 
lemma. We claim that the following three functions al- 
ways produce such sets: 

f /i(x)=Er=i3 2(j - 1) sign(3?{ a ;J) 
+ 3 2i - 1 sign(S{a; i }) 

' /2(x) = ||x|k=£LiM W 

f ( \ _ y-« msign(iR{a; i }) ■ S{xj} 

where 3?{.} and represent the real and imaginary 
parts, respectively, and msign{.) is the modified sign 
function which generates the same output as the sign(.) 
function except that msign(0) = 1. 
The first sampling function, /i, although produces only 
one sample, uniquely determines the sign of both real and 
imaginary parts of all the elements of x. Since the out- 
put range of sign function is restricted to three different 
values of —1, and 1, the sample generated by f\ can 
be viewed as the base 3 representation of an integer num- 
ber with the difference that the digit 2 is replaced with 
-1 (the same residue in division by 3 which yields the 
uniqueness). Moreover, this sample is an integer between 
— 3 ™~ x and 3 ~ x which requires \2n log 2 3] bits for er- 
rorless quantization. We will show that this sample guar- 
antees the left inequality. Let a and b be two unequal 
vectors that /i(a) = /i(b), we show that (a, b) 0: 

5R{(a,b}} = 5R{a ff • b} 

n 

= R{oi}K{6i} + 3{a 4 }3M (15) 

i=l 



Since /i(a) = /i(b), we have sign{?R{ai\) = 
sign(dl{bi}) and sign(^s{a,i}) = sign(^s{bi}) for all 
1 < i < n. Therefore »{a;} • + 9{aJ • > 

and the equality occurs if and only if 5R{ai} = = 
9{cii} = ^s{bi} = 0. Consequently we have: 

n 

H{<*i} ■ ^{h} + 3{<*i} • > 

i=l 

=^ 5R{(a,b}} > (16) 

with the equality for a = b = which contradicts the fact 
that a and b are unequal. Thus: 

{ AW=A(b) * s «- b »>» 

| (a, b) | > (17) 

Now using the samples generated by ft and / 3 (in ad- 
dition to fx) we prove the right inequality. If a and 
b are such that / 2 (a) = /2(b), / 3 (a) = / 3 (b) and 
I (a, b) I = ||a||^ 2 • ||b||£ 2 , a and b must have the same 
direction (Cauchy inequality) or equivalently, there exists 
c e C such that b = c a: 

n n 

/2(b) = / 2 (ca) = Y\c-ai\ = \c\Y\ a i\ 

i=l i=l 

= |c|/ 2 (a) (18) 

The condition /2(a) = /2(b) restricts the amplitude of 
c to have the value 1 (|c| = 1); note that /2(a) = re- 
sults in a = b = which again contradicts the previ- 
ous assumption of distinctness of a and b. Thus, up to 
this point we shown that by having the first two samples 
(generated by fi and / 2 ), the original vector within a con- 
stant complex coefficient on the unit circle in known. We 
show that the third sample uniquely determines the phase 
of that coefficient which implies the overall uniqueness. 
Before we proceed to reveal the role of the third sample, 
let us consider the effect of quantizing the second sample. 
Since the absolute-value operator is a continuous function, 
the sampling function / 2 is also a continuous function of 
the input vector. Continuity implies that small perturba- 
tions in the input such as quantization are mapped to small 
deviations in the output. Moreover, the quantization of 
this sample does not affect the mentioned restriction on c 
(\c\ — 1); i.e., the ambiguity is still the phase of c. In 
simple words, although the reconstructed amplitudes do 
not exactly match the original ones, the uniqueness (in the 
amplitude) is still valid. In fact, due to quantization, the re- 
produced amplitudes fall within a precision from the orig- 
inal values. If the input elements of the vector are quan- 
tized with I bits prior to sampling, the output of the / 2 can 
be quantized with I + |~log 2 (n)] bits (unsigned) without 
causing any further loss of information. 
Now we show the role of the last sample in revealing the 
phase of the nonzero elements of the input vector in the 
sparsity domain. Since the summation in / 3 is taken over 
the nonzero elements of the vector, the denominator in 
each of the terms is a positive real number (the denomina- 
tor becomes zero only when 5R{xi} = = which 



is excluded by Xi ^ 0). Besides, we have: 

msign(3fj{xi}) • %{xi} 



< 



|R{a*}| + 



\to{xi}\ + \$S{xi}\ 



< 1 



(19) 



which shows that sample generated by / 3 is bounded be- 
tween — n and n (boundedness is necessary for quantiza- 
tion). 

If a and b are two vectors which produce the same set of 
samples (all three), we know b = c a where |c| = 1 by 
use of the first two samples. Considering the third sample 
we should have /3(a) = /3(b): 



E 

= E 

cai^O 



msign(5R{ai}) • 3{ai} 
|3?{ ai }| + |3{oi}| 

msign(5R{cai}) • 3{cai} 



\^t{cai}\ + |3{cai}| 



(20) 



Owing to the fact that \c\ = 1, the conditions a t ^ 
and ca,i ^ are equivalent; therefore, the terms used 
in the summations are the same in both sides. In addi- 
tion, the first sample uniquely determines the sign and 
therefore msign of both real and imaginary parts; thus, 
we should have msign($t{ai\) = msign($t{cai}) and 
sign("<s{ai}) — sign(Q{cai}). To simplify the notations, 
we represent the real and imaginary parts of aj and cat by 
Ri,Ii and R^I^, respectively. Moreover, by sn and sn 
we mean the sign and msign operators: 

sn(Ri) ■ h sn(J? i ) • I t 



\Ri\ + \ii\ \r',\- 

= sn(Ri) ■ sn(Ii) 



I I 1 I I I 



= sn(Ri) ■ sn(Ii) 



Ri\ + \Ii\ \R' i \ + \l' i 
{\R i \ + \I i \){\R' i \ + \l' i \) 



h-Ri - h-Ri 



(m + \h\)(\R' l \ + %\) 

-S{c} ■ \a t \ 2 

{\R i \ + \i i \){\R' t \ + \r l \) 

Thus, / 3 (a) = / 3 (b) yields: 



(21) 



^0 W + \^)(\ R 'i\ + \ r i\) 

" v ' 

> 

=> 9{c} = 



(22) 



The above result in conjunction with the previous condi- 
tion that |c| = 1 remains two possible choices for c, 1 or 
— 1; nonetheless, c = — 1 produces different signs for real 
and imaginary parts in b which is not acceptable by the 
choice of the first sample. As a consequence, c = 1 is 
the only choice which means that a = b and the proof for 
the claimed unique 1 -sparse decodability of the aforemen- 
tioned set of sampling functions is complete. A similar ar- 
gument to the one presented for quantization of the second 



sample is valid here; even after quantization, there exists 
at most one 1 -sparse vector which produces these samples. 
Given that the quantization does not change the sign of the 
real and imaginary parts, the sampling functions will still 
be in their continuous region and therefore, small errors in 
the input (such as quantization) will be mapped to small 
errors in the reconstructed signal. 

It is easy to check that if the domain is R rather than C, 
the third sample is not required; i.e., the sign of the el- 
ements and the l\ norm of the vector suffice for unique 
representation of 1 -sparse vectors. 

5. Conclusions 

We have considered the deterministic sampling of sparse 
signals whose sparsity domain is not known at the time 
of sampling. Although random linear sampling of such 
signals has been shown to be a proper choice (under some 
conditions), the deterministic approach fails in reconstruc- 
tion of some identifiable subclasses. The mentioned draw- 
back is still an issue when nonlinear measurements of the 
fc-sparse signals where k > 1 are employed. It is shown 
that class of 1-sparse signals (in an arbitrary linear spar- 
sity domain) can be uniquely identified using nonlinear 
sampling. We have demonstrated a necessary and suffi- 
cient condition for the sampling functions to provide the 
unique 1-sparse decodability of the samples, in addition to 
presenting a realizable set of such functions. 
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